import CodeBlock from "@theme/CodeBlock";
import StreamIntermediateStepsExample from "@examples/agents/stream_intermediate_steps.ts";
import StreamEventsExample from "@examples/agents/stream_events.ts";
import StreamLogExample from "@examples/agents/stream_log.ts";

# Streaming

Streaming is an important UX consideration for LLM apps, and agents are no exception. Streaming with agents is made more complicated by the fact that it’s not just tokens that you will want to stream, but you may also want to stream back the intermediate steps an agent takes.

Let’s take a look at how to do this.

## Streaming intermediate steps

Let’s look at how to stream intermediate steps. We can do this by using the default `.stream()` method on the AgentExecutor.

import IntegrationInstallTooltip from "@mdx_components/integration_install_tooltip.mdx";

<IntegrationInstallTooltip></IntegrationInstallTooltip>

```bash npm2yarn
npm install @langchain/openai
```

<CodeBlock language="typescript">{StreamIntermediateStepsExample}</CodeBlock>

You can see that we get back a bunch of different information. There are two ways to work with this information:

1. By using the AgentAction or observation directly
2. By using the messages object

## Custom streaming with events

Use the `streamEvents` API in case the default behavior of stream does not work for your application (e.g., if you need to stream individual tokens from the agent or surface steps occuring within tools).

:::warning
This is a beta API, meaning that some details might change slightly in the future based on usage. You can pass a `version` parameter to tweak the behavior.
:::

Let’s use this API to stream the following events:

1. Agent Start with inputs
2. Tool Start with inputs
3. Tool End with outputs
4. Stream the agent final anwer token by token
5. Agent End with outputs

<CodeBlock language="typescript">{StreamEventsExample}</CodeBlock>

```
-----
Starting agent: Agent with input: {"input":"what is the weather in SF"}

-----
Starting tool: TavilySearchResults with inputs: weather in San Francisco

-----
Finished tool: TavilySearchResults

Tool output was: [{"title":"Weather in San Francisco","url":"https://www.weatherapi.com/","content":"Weather in San Francisco is {'location': {'name': 'San Francisco', 'region': 'California', 'country': 'United States of America', 'lat': 37.78, 'lon': -122.42, 'tz_id': 'America/Los_Angeles', 'localtime_epoch': 1707638479, 'localtime': '2024-02-11 0:01'}, 'current': {'last_updated_epoch': 1707638400, 'last_updated': '2024-02-11 00:00', 'temp_c': 11.1, 'temp_f': 52.0, 'is_day': 0, 'condition': {'text': 'Partly cloudy', 'icon': '//cdn.weatherapi.com/weather/64x64/night/116.png', 'code': 1003}, 'wind_mph': 9.4, 'wind_kph': 15.1, 'wind_degree': 270, 'wind_dir': 'W', 'pressure_mb': 1022.0, 'pressure_in': 30.18, 'precip_mm': 0.0, 'precip_in': 0.0, 'humidity': 83, 'cloud': 25, 'feelslike_c': 11.5, 'feelslike_f': 52.6, 'vis_km': 16.0, 'vis_miles': 9.0, 'uv': 1.0, 'gust_mph': 13.9, 'gust_kph': 22.3}}","score":0.98371,"raw_content":null},{"title":"San Francisco, California November 2024 Weather Forecast","url":"https://www.weathertab.com/en/c/e/11/united-states/california/san-francisco/","content":"Temperature Forecast Temperature Forecast Normal Avg High Temps 60 to 70 °F Avg Low Temps 45 to 55 °F Weather Forecast Legend WeatherTAB helps you plan activities on days with the least risk of rain. Our forecasts are not direct predictions of rain/snow. Not all risky days will have rain/snow.","score":0.9517,"raw_content":null},{"title":"Past Weather in San Francisco, California, USA — Yesterday or Further Back","url":"https://www.timeanddate.com/weather/usa/san-francisco/historic","content":"Past Weather in San Francisco, California, USA — Yesterday and Last 2 Weeks. Weather. Time Zone. DST Changes. Sun & Moon. Weather Today Weather Hourly 14 Day Forecast Yesterday/Past Weather Climate (Averages) Currently: 52 °F. Light rain. Overcast.","score":0.945,"raw_content":null},{"title":"San Francisco, California February 2024 Weather Forecast - detailed","url":"https://www.weathertab.com/en/g/e/02/united-states/california/san-francisco/","content":"Free Long Range Weather Forecast for San Francisco, California February 2024. Detailed graphs of monthly weather forecast, temperatures, and degree days.","score":0.92177,"raw_content":null},{"title":"San Francisco Weather in 2024 - extremeweatherwatch.com","url":"https://www.extremeweatherwatch.com/cities/san-francisco/year-2024","content":"Year: What's the hottest temperature in San Francisco so far this year? As of February 2, the highest temperature recorded in San Francisco, California in 2024 is 73 °F which happened on January 29. Highest Temperatures: All-Time By Year Highest Temperatures in San Francisco in 2024 What's the coldest temperature in San Francisco so far this year?","score":0.91598,"raw_content":null}]

-----
| The
|  current
|  weather
|  in
|  San
|  Francisco
|  is
|  partly
|  cloudy
|  with
|  a
|  temperature
|  of
|
| 52
| .
| 0
| °F
|  (
| 11
| .
| 1
| °C
| ).
|  The
|  wind
|  speed
|  is
|
| 15
| .
| 1
|  k
| ph
|  coming
|  from
|  the
|  west
| ,
|  and
|  the
|  humidity
|  is
|  at
|
| 83
| %.
|  If
|  you
|  need
|  more
|  detailed
|  information
| ,
|  you
|  can
|  visit
|  [
| Weather
|  in
|  San
|  Francisco
| ](
| https
| ://
| www
| .weather
| api
| .com
| /
| ).

-----
Finished agent: Agent

Agent output was: The current weather in San Francisco is partly cloudy with a temperature of 52.0°F (11.1°C). The wind speed is 15.1 kph coming from the west, and the humidity is at 83%. If you need more detailed information, you can visit [Weather in San Francisco](https://www.weatherapi.com/).

-----
```

## Other approaches

### `streamLog`

You can also use the astream_log API. This API produces a granular log of all events that occur during execution. The log format is based on the [JSONPatch](https://jsonpatch.com/) standard.
It’s granular, but requires effort to parse. For this reason, we created the `streamEvents` API as an easier alternative.

In addition to streaming the final result, you can also stream tokens from each individual step. This will require more complex parsing of the logs.

Note: You will also need to make sure you set the LLM to return streaming output to get the maximum amount of data possible.

<CodeBlock language="typescript">{StreamLogExample}</CodeBlock>

With some creative parsing, this can be useful for e.g. streaming back just the final response from the agent:

```typescript
const logStream = await agentExecutor.streamLog({
  input: "what is the weather in SF",
});

/*
  Final streamed output from the OpenAI functions agent will look similar to the below chunk
  since intermediate steps are streamed functions rather than strings:

  {
    "ops": [
      {
        "op": "add",
        "path": "/logs/ChatOpenAI:2/streamed_output_str/-",
        "value": "anc"
      }
    ]
  }
*/

for await (const chunk of logStream) {
  if (chunk.ops?.length > 0 && chunk.ops[0].op === "add") {
    const addOp = chunk.ops[0];
    if (
      addOp.path.startsWith("/logs/ChatOpenAI") &&
      typeof addOp.value === "string" &&
      addOp.value.length
    ) {
      console.log(addOp.value);
    }
  }
}

/*
  The
  current
  weather
  in
  San
  Francisco
  is
  
  52
  °F
  with
  broken
  clouds
  .
  There
  is
  a
  chance
  of
  rain
  showers
  with
  a
  low
  of
  
  54
  °F
  .
  Winds
  are
  expected
  to
  be
  from
  the
  SSE
  at
  
  5
  to
  
  10
  mph
  .
  For
  more
  detailed
  information
  ,
  you
  can
  visit
  [
  Weather
  Underground
  ](
  https
  ://
  www
  .w
  under
  ground
  .com
  /h
  our
  ly
  /us
  /ca
  /s
  an
  -fr
  anc
  isco
  /date
  /
  202
  3
  -
  12
  -
  27
  ).
*/
```
